Add configure, configure Makevars.in -> Makevars #165

mlondschien · 2024-01-07T18:44:53Z

No description provided.

changeforest-r/src/Makevars.in

Ilia-Kosenkov · 2024-01-07T19:22:59Z

I think some one else is missing, unable to cross-compile. Lemme check

Ilia-Kosenkov · 2024-01-07T19:34:46Z

@eitsupi, hi, we are dealing with a cross-compilation issue. Could you please help us figure out cross-compilation from Max OS x86_64 to aarch64? I used polars-r as an example, but I still do not understand how to build a package without linking to R library, which in this case causes an architecture mismatch (R session links to x86_64 library while cargo artifact is aarch64)

eitsupi · 2024-01-07T23:07:56Z

I don't know how to cross compile R packages.
Regarding conda-forge, packages like dplyr have binaries for arm64, so why not use them as a reference?

The only thing I know a little about is cross-compilation in Rust. I'm sure you can find a lot of information about it on the internet.

yutannihilation · 2024-01-08T01:16:29Z

Okay, so, if even polars is not cross-compile-aware, I guess there's no Rust-powered R package that does it correctly. In my understanding, extendr never aimed at cross-compiling for arm64 macOS, so it's no surprise that this doesn't work. Anyone would be welcomed to explore this frontier. But, IMHO, cross-compiling is a temporary solution until the arm64 macOS runner is generally available, so I'm not sure if it's really worth the effort.

That being said, here's a good news (or bad news) that we might get a reason that forces us to think about cross-compiling seriously.

r-universe-org/help#97 (comment)

eitsupi · 2024-01-08T01:34:49Z

What is possible with prqlr (and polars) is to cross-compile the Rust library in advance and use that binary when installing the R package to avoid the cargo build when installing the R package.
https://github.com/eitsupi/prqlr/blob/76c68dda761c0b5f5783b03154c20b6127b306b1/configure#L50-L76
https://github.com/eitsupi/prqlr/blob/76c68dda761c0b5f5783b03154c20b6127b306b1/src/Makevars.in#L19-L22
We have never cross-compiled the entire R package.

So if a cross-compilation method has been established for C, it would be possible to build the R package by cross-compiling Rust and C in this way, respectively.

As I mentioned above, some packages such as dplyr are available on arm64 Linux and macOS on conda-forge, so it is not impossible to build R packages using Rust, I think?

yutannihilation · 2024-01-08T01:51:02Z

Yeah, I believe it's definitely possible and I bet it's just specifying target correctly.

eitsupi · 2024-01-08T01:58:44Z

As for Linux, I believe we can build the R package by cross-compiling the Rust library on the host machine and then running arm64 R on Docker with qemu.
But not sure about macOS.

Since cargo-cross seems to support cross-compilation of packages other than Linux, it may be possible to cross-compile R packages in the same way, but I have not tried it.
(Incidentally, since there is no R on the Docker image used by cargo-cross, it cannot be used for Linux as it is.)
https://github.com/cross-rs/cross

yutannihilation · 2024-01-08T06:14:54Z

Confirmed that specifying --target makes it possible to cross-compile for arm64 macOS at least on R-universe. I'm not interested in conda so I'd stop here, but hope this helps to some extent.

diff: https://github.com/yutannihilation/string2path/compare/912ae92..9576797
log: https://github.com/r-universe/yutannihilation/actions/runs/7443083907/job/20247512893#step:7:56

mlondschien · 2024-01-08T08:48:55Z

I get the same error locally:

    Finished release [optimized] target(s) in 1m 57s
if [ "" != "true" ]; then \
                rm -Rf /Users/mlondschien/code/changeforest/changeforest-r/src/.cargo && \
                rm -Rf ./rust/target/release/build; \
        fi
arm64-apple-darwin20.0.0-clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/mlondschien/mambaforge/envs/r-test/lib/R/lib -Wl,-dead_strip_dylibs -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/mlondschien/mambaforge/envs/r-test/lib -L/Users/mlondschien/mambaforge/envs/r-test/lib -o changeforest.dylib entrypoint.o -L./rust/target/release -lchangeforestr -L/Users/mlondschien/mambaforge/envs/r-test/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
ld: library not found for -lchangeforestr
arm64-apple-darwin20.0: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [changeforest.dylib] Error 1
ERROR: compilation failed for package ‘changeforest’
* removing ‘/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/changeforest’
* restoring previous ‘/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/changeforest’

If I replace -lchangeforestr with -lchangeforest in the Makevars, I get

arm64-apple-darwin20.0.0-clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/mlondschien/mambaforge/envs/r-test/lib/R/lib -Wl,-dead_strip_dylibs -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/mlondschien/mambaforge/envs/r-test/lib -L/Users/mlondschien/mambaforge/envs/r-test/lib -o changeforest.dylib entrypoint.o -L./rust/target/release -lchangeforest -L/Users/mlondschien/mambaforge/envs/r-test/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
ld: warning: -pie being ignored. It is only used when linking a main executable
installing to /Users/mlondschien/mambaforge/envs/r-test/lib/R/library/00LOCK-changeforest-r/00new/changeforest/libs
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘changeforest’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/00LOCK-changeforest-r/00new/changeforest/libs/changeforest.dylib':
  dlopen(/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/00LOCK-changeforest-r/00new/changeforest/libs/changeforest.dylib, 0x0006): symbol not found in flat namespace '_R_init_changeforest_extendr'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/changeforest’
* restoring previous ‘/Users/mlondschien/mambaforge/envs/r-test/lib/R/library/changeforest’

yutannihilation · 2024-01-08T09:31:42Z

If you specify --target=aarch64-apple-darwin, LIBDIR should be ./rust/target/aarch64-apple-darwin/release, not ./rust/target/release.

arm64-apple-darwin20.0.0-clang ..snip.. -L./rust/target/release -lchangeforestr ...

mlondschien added 2 commits January 7, 2024 19:44

Add configure, configure Makevars.in -> Makevars

822c845

Empty Makevars.

52c61fd

mlondschien mentioned this pull request Jan 7, 2024

Linking issue on osx-arm64 conda-forge/r-changeforest-feedstock#28

Closed

1 task

Ilia-Kosenkov reviewed Jan 7, 2024

View reviewed changes

changeforest-r/src/Makevars.in Outdated Show resolved Hide resolved

This comment was marked as off-topic.

Sign in to view

eitsupi mentioned this pull request Jan 8, 2024

Support arm64 cross-compile on R-universe PRQL/prqlc-r#246

Closed

mlondschien added 2 commits January 8, 2024 12:29

Different permissions for configure.

9272bc7

Implement suggestion by @yutannihilation

d931494

mlondschien force-pushed the use-polars-makewin-and-configure branch from 84c3495 to d931494 Compare January 8, 2024 11:34

mlondschien merged commit f2159e9 into main Jan 8, 2024
20 checks passed

mlondschien deleted the use-polars-makewin-and-configure branch January 8, 2024 12:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add configure, configure Makevars.in -> Makevars #165

Add configure, configure Makevars.in -> Makevars #165

mlondschien commented Jan 7, 2024

Ilia-Kosenkov commented Jan 7, 2024

Ilia-Kosenkov commented Jan 7, 2024 •

edited

Loading

eitsupi commented Jan 7, 2024

yutannihilation commented Jan 8, 2024

eitsupi commented Jan 8, 2024 •

edited

Loading

yutannihilation commented Jan 8, 2024

eitsupi commented Jan 8, 2024

This comment was marked as off-topic.

yutannihilation commented Jan 8, 2024

mlondschien commented Jan 8, 2024

yutannihilation commented Jan 8, 2024

Add configure, configure Makevars.in -> Makevars #165

Add configure, configure Makevars.in -> Makevars #165

Conversation

mlondschien commented Jan 7, 2024

Ilia-Kosenkov commented Jan 7, 2024

Ilia-Kosenkov commented Jan 7, 2024 • edited Loading

eitsupi commented Jan 7, 2024

yutannihilation commented Jan 8, 2024

eitsupi commented Jan 8, 2024 • edited Loading

yutannihilation commented Jan 8, 2024

eitsupi commented Jan 8, 2024

This comment was marked as off-topic.

yutannihilation commented Jan 8, 2024

mlondschien commented Jan 8, 2024

yutannihilation commented Jan 8, 2024

Ilia-Kosenkov commented Jan 7, 2024 •

edited

Loading

eitsupi commented Jan 8, 2024 •

edited

Loading